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1. Introduction 

The goal of this paper is to define new deformations of group algebras 
^ . of Coxeter groups. Recall that a Coxeter group W is generated by ele- 

^w' I ments Si,i € / modulo two kinds of relations - the involutivity relations 

^ ' •sf = 1 and the relations (sjSj)"^'^ = 1, where 2 < niij = rriji < oo; in pres- 

"j^ [ ence of the involutivity relations these are equivalent to the braid relations 

SiSjSi... = SjSiSj... [rriij factors). The traditional way to deform the group 
algebra C[VF] is to deform the involutivity relations to the Hecke relations 
(sj — q){si + q^^) = 0, and keep the braid relations unchanged. This yields 
the Hecke algebra Cg[M^] of W ^ which is classically known to be a flat de- 
formation of C[VF]. 

On the contrary, the deformation A of C[Ty] that we study in this paper 

■ is obtained by keeping the involutivity relations fixed, and deforming the 
\ braid relations to 

when ruij < oo. Here tijk, k G Z^-^. , are new commuting variables such that 
tijk = i^i-ki ^"^^ Sptijk = tjikSp. We also consider the subalgebra of A 
generated by SiSj and tijk- It is a deformation of the group algebra C[W4.] 
of the group W+ of even elements of W. ^ 

A priori, it is not clear that the algebras A, A^ are "well behaved" . We 
^ ■ show that their "good" or "bad" behavior is determined completely by 

^ I whether they are formally flat (i.e. whether the corresponding completed al- 

gebras over C[[logtjjfc]] are flat deformations of C[T^], C[W+]). More specifi- 
cally, we show that if A is formally flat, then it is algebraically flat (i.e., free 
as a module over R := C[tjjfc]), and moreover any set of reduced words in 
Si bijectively representing all elements of W defines a basis in A over R. In 
particular, this yields a canonical filtration F' on A (by length of reduced 
words) such that F'^A/F'^-^A is a finitely generated free i?-module. 

Unlike the Hecke algebra Cq[M^], the algebras A, A^ are not necessarily 
flat, and we determine when exactly this happens. First of all, it is easy to 
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^In view of the relation Sptijk = tjikSp, A is not quite an honest deformation of C[W] (as 
the variables tijk are not central). However, the subalgebra A+ is an honest deformation 
of C[W+], and ^4 is a semidirect product of Z2 with A+. 
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see that flatness always holds for Coxeter groups of rank 1 and 2. Further, 
we show that for Coxeter groups of rank 3, the flatness of A, is equivalent 
to the condition that W is an infinite group. In other words, the (unordered) 
triple mi2,mi3, 771-23 must be different from the triples (2,2, m) (m < oo), 
(2, 3, 3), (2, 3, 4), (2, 3, 5), which correspond to finite Coxeter groups of rank 

3: Ai X /2m, A3,B3,H3. 

This implies that in any rank, a necessary condition for A, A^ to be flat 
is that W does not contain finite parabolic subgroups of rank 3. Our main 
result asserts that this condition is also sufficient. The proof is based on 
consideration of constructible sheaves on the cell complex associated to the 
group W. 

The motivation for this paper comes from the fact that for rank 3 Coxeter 
groups W, the algebra coincides with the Hecke algebra corresponding to 
the orbifold H/F, where H is the sphere, Euclidean plane, or Lobachevsky 
plane, and F is the group generated by rotations by the angles 27r/mjj 
around the vertices of a triangle in H with angles vr/mjj. Such Hecke al- 
gebras were introduced in 0, and it was shown in [Ej (using the theory of 
Cherednik algebras and KZ functor) that they are formally flat if // is a 
Euclidean or Lobachevsky plane, but not flat if is a sphere, which is our 
main result for rank 3. (In fact, the proof of the main result in arbitrary 
rank is based on similar ideas, the difference being that the category of D- 
modules used in [E] is replaced with the category of constructible sheaves.) 
Thus, as a by-product, we obtain the algebraic PBW theorem for the Hecke 
algebras of polygonal Fuchsian groups defined in 

The cases when H is the Euclidean plane and niij < oo (i.e., {mi2,mi3, 17123) = 
(3, 3, 3), (2, 4, 4), (2, 3, 6), an d W is the afhne Weyl group of types A2, B2, 
G2) were also discussed in |E()R) . as in these cases the algebras ^+ are 
the generalized double affine Hecke algebras of rank 1 of types Eq,Ey,Es 
which provide quantizations of del Pezzo surfaces. Thus we obtain new 
PBW filtrations and bases on these algebras, which (unlike those in jEOR ) 
are constructed without using a computer. (These filtrations, however, have 
the flaw that the idempotent used to define the spherical subalgebra is not 
homogeneous) . 

Acknowledgments. P.E. is very grateful to A. Polishchuk for an expla- 
nation about constructible sheaves, and thanks A. Braverman, D. Kazhdan, 
and C. McMullen for useful discussions. The work of P.E. was partially 
supported by the NSF grant DMS-9988796 and the CEDE grant RMl-2545- 
MO-03. E.R. was supported in part by NSF Grant No. DMS-0401387. 

2. Definition of the algebras A{M),A+{M) 

2.1. Coxeter groups. Recall the basics of the theory of Coxeter groups 
(see e.g. |Bj). 

Let / be a finite set. Let Z>2 denote the set of integers which are > 2. 
A Coxeter matrix over / is a collection M of elements niij € Z>2 U {00}, 
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i,j € I, i ^ j, such that rriij = rriji. The rank r of M is, by definition, the 
cardinahty of the set /. 

Let M be a Coxeter matrix. Then one defines the Coxeter group ^ W{M) 
by generators Si, i & I, and defining relations 

sf = 1, {siSj)"^'^ = 1 if rriij / oo. 

The group W{M) has a sign character ^ : W{M) — > {±1} given by ^(sj) = 
— 1. Denote by VF+(M) the kernel of ^, i.e. the subgroup of even elements. 
Let /' C / be a subset, and M' the submatrix of M consisting of rriij £ 
Then we have a natural map W{M') — > W{M). It is known ((HI) that 
this map is injective. Thus W{M') is a subgroup of W{M), which is called 
the parabolic subgroup corresponding to /'. 

2.2. Deformed Coxeter group algebras. Define the algebra A{M) by 
invertible generators Si,i € /, and tijk, h j ^ I, k ^ Z^^^ for such that 
rriij < oo with defining relations 

tijk = tji_k^ Si = 1, 

rriij 

W{siSj - tijk) = if rriij < oo, 

k=l 

[tijkjti'j'k'] — 0, Sptijk — tjikSp. 

Define also the algebra A^{M) over R := C[tijk] {tijk = tjj^-k) gener- 
ators Qij, i ^ j {aij = aji^), and relations 

rriij 

Y\_iaij - tijk) = if rriij < oo, 

k=l 

(rijQ>jp(Xpi — 1. 

Let € / be an element. Then we can define an involutive automorphism 
(To of A_|_(M) (as an algebra over C) by the formulas o-o(tjjfc) = tjik, (JQ{aij) = 
aji if i = or j = 0, and (To(ajj) = aoio^jo if h3 7^ 0. It is easy to show 
that this automorphism is well defined.^ Thus we can define the semidirect 
product Aq{M) = CL2 X ^+(-^) using the automorphism ctq- 

Proposition 2.1. The assignment f{aij) = SiSj, /{(Tq) = sq uniquely ex- 
tends to an isomorphism f : Aq{M) A{M). 

Proof. It is easy to check that / uniquely extends to a surjective homomor- 
phism of algebras. Moreover, it is easy to construct the inverse of /: it is 
given by the formula f'^{si) = aoa^i for i ^ 0, and /(sq) = (^o- We are 
done. □ 

When we talk about Coxeter groups, we always assume that they are equipped with 
a fixed system of generators, which corresponds to the notion of a Coxeter system from 

m 

Note that up to inner automorphisms, ao does not depend on the choice of the element 
G /. 
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Thus, A{M) = CZ2 X A+{M). 

The following proposition explains the connection between the algebras 
A{M), A+{M) and the groups WiM), W+{M). 

Let J be the ideal in R generated by the elements tijk — exp{2TTik/mij). 
It is easy to see that JA{M) = A{M)J, so JA{M) is a two-sided ideal in 
A{M). 

Proposition 2.2. One has A{M)/JA{M) = C[W{M)], A+{M)/JA+{M) = 
C[W+{M)]. 

Proof. Straightforward. □ 

2.3. Spanning sets for A{M), A^{M). If ti; is a word in letters Sj, let 
be the corresponding element of A{M). Choose a function w{x) which 
attaches to every element x € W{M), a reduced word w^x) representing x 
in W{M). 

We will now prove the following important result. 

Theorem 2.3. (i) The elements T^(^), x £ W, form a spanning set in 
A{A1) as a left R-module. 

(ii) The elements T^(x); x £ W^, form a spanning set in A^{M) as a left 
R-module. 

Proof. It is clear that (ii) follows from (i), so it suffices to prove (i). 
Let us write the relation 

rriij 

Y\_iSiSj - tijk) = 
k=l 

as a deformed braid relation: 

SjS'iSj... ~\~ S.L.T. — tijSiSjSi... ~\~ S. L.T.J 

where tij = {—l)^'^~^^tiji...tijmij^ S.L.T. mean "smaller length terms", and 
the products on both sides have length niij. This can be done by multiplying 
the relation by SiSj... {rriij factors). 

Now let us show that T^(^) span A(M) over R. Clearly, T^ for all words 
w span A{M). So we just need to take any word w and express T^ via T^i^^y 

It is well known from the theory of Coxeter groups (see e.g. jB]) that 
using the braid relations, one can turn any non-reduced word into a word 
that is not square free, and any reduced expression of a given element of 
W{M) into any other reduced expression of the same element. Thus, if w is 
non-reduced, then by using the deformed braid relations we can reduce T^ 
to a linear combination of T„ with words u of smaller length than w. On 
the other hand, if tt; is a reduced expression for some element x G W ^ then 
using the deformed braid relations we can reduce to a linear combination 
of T„ with u shorter than and T^^^^). Thus T^i^^^-^ are a spanning set. The 
theorem is proved. □ 

Thus, Aj^[M) is a "deformation" of C[VF+(M)] over i?, and similarly 
A{M) is a "twisted deformation" of C[W{M)]. 
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3. Flat Coxeter matrices 

3.1. Definition of a flat Coxeter matrix. Denote by A{M), A^{M) the 
formal versions of A{M), A^{M), i.e., algebras generated (topologically) by 
the same generators and relations, but with tijk = e^'^^^/^ij ^^iik ^ where Tijk 
are formal parameters. By virtue of Theorem l2.31 Aj^{M) is an algebra over 
R := C[[Tijk]] which is a formal deformation of C[H^+] with deformation 
parameters Tijk- 

Definition 3.1. We say that M is a flat Coxeter matrix if A^{M) is a flat 
deformation of C[W^-f], i.e. if A^{M) is a topologically free left i?-module. 

Since A(AI) = CZ2 ix A-|_(M), for a flat Coxeter matrix we also have that 
A{M) is topologically free as a left ij-module. 

3.2. Bases oi A{M),A+{M) for fiat M. 

Proposition 3.2. Let M be a flat Coxeter matrix. Then 

(i) The elements T^(x); x € W , form a basis in A{M) as a left R-module. 
(a) The elements T^(^.j.^, x G form a basis in A^{M) as a left R- 

module. 

Proof. It is sufficient to show that T^^^^ are linearly independent. This 

follows from the fact that they are linearly independent in A{M), which is 
a consequence of the flatness of M. □ 

Corollary 3.3. The R-modules A{M) and Aj^{M) are free. Moreover, they 
carry a filtration F* , defined by the condition that F"'A(M) is spanned by 
Tw{x) for X of length < n. This filtration has the property that the R-modules 
gr„A(M) and gr^A+{M) are finitely generated and free. 

We note that the filtration F* is canonical, i.e., independent on the choice 
of the function w(x). 

Remark. Let T{M) be the graph whose vertices are elements of I, and 
i,j are connected if rriij is odd. Then the filtration F^ can be refined to a 
multi- filtration F", where n is a nonnegative integer function on / which 
is constant on the connected components of T(M). Namely, F^A(M) is 
spanned by T^(^,j.-j with w{x) involving < n(i) copies of Si for each i. It is 
easy to see from the above that gr^A{M) is finitely generated and free over 
R. 

3.3. When is a Coxeter matrix fiat? Let us study the question when 
a given Coxeter matrix is flat (i.e., when the algebras A{M), A-^-(M) are 
"meaningful" ) . 

The case of rank 1 is trivial. In rank 2, the algebra A^{M) is C[a,a~^] 
if mi2 = 00 and C[a,a^^]/{P{a)) where P{a) = (a — ii2i)...(a — ti2mi2) 
otherwise. Thus, any Coxeter matrix of rank 2 is flat. 

However, in rank 3, we have a much more interesting situation. Namely, 
we have the following theorem. 
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Theorem 3.4. A Coxeter matrix M of rank 3 is flat if and only if the group 
W{M) is infinite. 

Coxeter matrices of rank 3 are conveniently written as triples of numbers 
mi2, ""1.23) ^31 (the order does not matter). Recall that the Coxeter matrices 
of rank 3 producing a finite Coxeter group are the following: 

1. M = (2,2,?7i), m < oo (type Ai x /2m)- 

2. M = (2,3,3) (type ^3) 

3. M = (2,3,4) (type B3) 

4. M = (2,3,5) (typeFs) 

Thus, the theorem claims that a Coxeter matrix is flat if and only if it 
does not fall into the four cases listed above. 

Proof. If. Let M = {mi2,m23,m3i). The algebra A^{M) is defined over R 
by generators 012,023,031 with defining relations 

(Ojj tjji)...((2jj tijrriij) — 0) ij — 12,23,31, 

and 

012023031 = 1- 

This algebra is a deformation of the group algebra of the group F = -/'mi2,m23,m3i 
generated by 012,023,031 with defining relations Oj^*^ = 1, 012O23O31 = 1. 
The group F is isomorphic to the group generated by rotations by an- 
gles 2'K/mij around vertices of a triangle whose angles are vr/mjj. This 
triangle lies on the sphere, plane, and Lobachevsky plane if the quantity 
S = — h — h — is > 1,= 1, and < 1, respectively. The cases 1,2,3,4 
when W{M) is finite correspond exactly to the case S > 1. Now, the flat- 
ness of the algebra j4+(M) for S < 1 follows from Theorem 3.3 of [E| (as 
Aj^{M) is the Hecke algebra of the orbifold H/F, where H is the plane or 
Lobachevsky plane, depending on whether = 1 or 5 < 1). Another proof 
of flatness of A^{M) for 5 = 1 is given in jEORj (in this case A_|_(M) is a 
generalized double affine Hecke algebra). This proves the "if" direction of 
the theorem. 

Only if. Suppose W{M) is finite. Assume the contrary, i.e., that M is 
fiat. Then the algebra A^{M) is a free module over R of dimension D = 
|W+(-/Vf)|. The eigenvalues of Ojj in the regular representation are equal to 
tijk, and occur with multiplicity D/ruij. Thus in the regular representation 
we have det(ojj) = (J^;, tijk)^^"^^^ . So taking the determinant of the relation 
012O23O31 = 1, we have 

mi2 m23 rn-ii 

(n*i2-fe)''^"''(n*23fc)''^"''(n*3ifc)''/™^^ = i. 

k=l k=l k=l 

This is a nontrivial relation on tijk, which contradicts to the fiatness of M. 
The theorem is proved. □ 



Now we are ready to state our main result. 
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Theorem 3.5. A Coxeter matrix M is flat if and only if for any 3-element 
subset {i,j,k} of I, the Coxeter group generated by Si,Sj,Sk is infinite. 

The theorem is proved in the next susbection. Note that in the process 
of proving Theorem 13.51 we give a new proof of the "if" part of Theorem 
13.41 which rehes on arguments from topology (constructible sheaves) rather 
than complex analysis (D-modules used in ^). 

3.4. Proof of Theorem 13.51 Let us first prove the easier "only if" direc- 
tion. Let I' = {i,j,k} C /. Let M' be the corresponding Coxeter ma- 
trix. Since W{M') C W{M), the flatness of A+{M) imphes the flatness of 
A+(M'). But if A+{M') is flat then by Theorem lOl W{M') is an inflnite 
group, as desired. 

Now let us prove the "if" direction. To do so, we introduce a 2-dimensional 
cell complex E attached to M as follows. The zero-dimensional cells of S are 
the elements of 1^ := W{M). The 1-dimensional cells are edges connecting 
w and SiW for each z G /. Then we have cycles w, SiW, SjSiW, SiSjSiW..., SjW 
of length 2mij (if niij < oo), and the 2-dimensional cells of S are 2mjj-gons 
attached to these cycles. 

It is easy to see that W acts properly discontinuously on E, hence so does 
its subgroup W^. Moreover, it is clear that the only flxed points of the 
VF+-action on E are the centers of the 2mjj-gons, with stabilizer Z^. .. Thus 
we can define an orbifold cell complex Y := 'S/W^. It has two vertices (the 
north and south pole, A^, S), edges between them corresponding to Si, and 
for each 2-element set C /, a disk Dij whose boundary is identified 

with the circle made up by Cj and ej. The disk Dij has an orbifold point Zij 
in the center, whose isotropy group is Z^i^ . 

We will need the following theorem (see e.g. |DMj ) . 

Theorem 3.6. // W has no finite parabolic subgroups of rank 3 then E is 
contractible. 

Let C be the category of constructible sheaves of complex vector spaces 
on E with respect to the stratification into cells (see (Sch) for definitions). 
Let C be the constant sheaf on E. 

Lemma 3.7. For any Coxeter matrix we have Ext^(C,C) = -fr^(E,C), and 
Ext^(C,C) C H^i^X)- 

Proof. ^ Let C be the abelian category of all sheaves of complex vector 
spaces on E. Then C is full abelian subcategory closed under extensions. 
In this situation for any F,G £ C, Ext^(-F, G) = Ext~(F, G), and the nat- 
ural map Ext^(-F, G) — > Ext~(-F, G) is injective. But it is well known that 
ExtUC, C) = F*(E, C). This implies the statement. □ 



This argument was provided to us by A. Polishchuk. 

7 



Remark. As was explained to us by A. Polishchuk, the inclusion Ext^(F, G) 
Ext~(-F, G) is in fact an isomorphism. It follows from the fact that our strat- 
ification satisfies the property that the closure of every cell is homeomorphic 
to a closed ball (in a way compatible with boundaries). Indeed, it suffices to 
prove this when F and G are simple. Then it follows from the fact that the 
algebra of Ext- between simple objects of C is generated in degree 1 (apply 
Corollary 2.2 of [P] for zero perversity). 

Lemma 3.8. If W has no finite parabolic subgroups of rank 3, one has 
Ext^(C,C) = for j = 1,2. 

Proof. The result follows from Theorem 13.61 and Lemma 13.71 □ 

Let V be the category of constructible sheaves on the orbifold Y with 
respect to stratification into cells. Thus, an object of P is a constructible 
sheaf on the complement y' C y of the points Zy with respect to the same 
stratification (intersected with Y'), such that the monodromy gij around Zij 
satisfies the equation g^^' = 1. 

Let TT : S — > y be the natural projection, vri : C ^ T> he the direct 
image functor with compact supports, and M = 7r!(C). Thus M is a local 
system on the orbifold Y. The monodromy representation of M over Y' is 
the regular representation of 

Lemma 3.9. One has Ext^M, M) = for j = 1, 2. 
Proof. 

Ext^(M,M) = Exty7r!C,7r!C) = 
Ext^(C, 7r*7r!C) = C[W+] Ext^(C, C). 
But Ext^(C, C) = by Lemma ESI We are done. □ 

A basic fact about constructible sheaves, going back to Fulton, Goresky, 
McCrory, and MacPherson ( |Shj : see also | Vy| ) , is that the category of 
constructible sheaves on a cell complex with respect to the stratification 
into cells is equivalent to the category of "cellular sheaves", i.e., modules 
over a certain algebra B (path algebra of a certain quiver with relations). 

Let us construct B in the case of the orbifold cell complex Y. Let us fix 
an ordering on the set I. Let S G V. Let V^, Vs, Vi, Vij {i ^ j) be the 
spaces of sections of S over small neighborhoods of iV, 5, the midpoints of 
Cj, and the points of Dij close to the midpoints of Cj, respectively. We then 
have natural restriction maps fNi '■ Vn —f Vi, fsi : V5 ^ T^, hij : Vi Vij, 
and monodromy maps gij : Vij Vji, with relations 

QijhijfNi = gjifNj, for i < j, 

Oijhijfsi = gjifsj, for i > j, 

and 
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Thus, S defines a representation of a certain quiver witli relations. We define 
the algebra B to be the path algebra of this quiver (modulo the relations). 
It is then well known that category T> is equivalent to the category of B- 
modules: the equivalence sends S to the i?-module 

V ■.= VN®Vse QiVi e eijVij. 

Now let T = {Tijk), k G ^rriij) be a collection of formal parameters. Define 
the algebra B{t) by the same generators as B, and the same relations except 
for one modification: the relation {gijgji)™"'^ = 1 is replaced with 

{g - tiji)...{g - tij^m.j) = 0, 

where g := gijgji- (we recall that tij^ := g^'^^'^/™''-'' e'^'-'* ) . It is clear that 
B{t)/{t = 0) = B. 

We will need the following flatness result. 

Proposition 3.10. For any Coxeter matrix M , the algebra B{t) is a flat 
deformation of B . 

Proof. For any i < j let pij be the idempotent of B which acts by 1 on 
VN,Vs,Vi,Vj and on Vij,Vji if ruij < oo, and acts by on all the other 
spaces. Then the direct sum of right modules pijB over B is faithful, so it 
suffices to show that they can be deformed to right i?(T)-modules. Clearly, 
B preserves the kernel of pij, so PijBB = pijBpij, and thus Bij := pijB is 
a unital algebra with unit pij. Replacing relations as above, we can define 
a deformation Bij^r) = pijB{T)pij of and it suffices to show that this 
deformation is flat. 

We clearly only need to consider the case niij < oo. Let us consider 
the category £ij of representations of Bij in which all the quiver arrows are 
isomorphisms. It is easy to see (by explicitly writing a basis of Bij) that 
the direct sum of these representations is faithful. Thus it suffices to show 
that any such representation can be deformed to a representation of -Bjj(r). 
But it is easy to see that the category £ij is equivalent to the category 
of vector spaces U with a linear map g : U ^ U such that g'™'-'' = 1. 
This shows that any object of this category can be easily deformed to a 
module over Bij^r) (by deforming g to an operator satisfying the equation 
{g - tiji)---{g - tijm.j) = 0), as desired. □ 

Now we can finish the proof of Theorem 11151 We can regard M as a 
S-module (in which all the arrows are isomorphisms). By Lemma VAM 
Ext^(M,M) = Ext|(M,M) = 0. This implies that M can be uniquely 
deformed to a module M,- over Br- The module M,- is a quiver represen- 
tation where all arrows are isomorphisms, so it represents a local system on 
Y' (over C[[t]]). The monodromy of this local system is a representation of 
A^(M), deforming the regular representation of C[II^+]. The existence of 
such deformation implies the flatness of A^{M). Theorem 18.51 is proved. 
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3.5. Flatness of Hecke algebras of polygonal Fuchsian groups. Let 

r = r(mi, ...,mr), r > 3, be the Fuchsian group defined by generators cj, 
j = 1, ...,r, with defining relations 



r 



Here 2 < mj < oo. 

In ^ the first author defined the Hecke algebra of T, 7^(r), by the same 
(invertible) generators and relations 

r 

{cj - tji)...{cj - tjm,) = 0, if rrij < oo, Cj = 1, 
where tj^ are invertible variables. 

Theorem 3.11. The algebraTi.{T) is free as a left module over R := C[t,t^"'^] 
if and only — l/rrij) > 2 (i.e., T is Euclidean or hyperbolic). 

The formal version of this theorem is proved in [E] . 

Proof. Let M be the Coxeter matrix of rank r such that mj_j+i := rui for 
i (z Ijr, and ruij = oo otherwise. It is easy to deduce from Theorem 13.51 
that M is fiat if and only if ^^(1 — ^/''^j) ^ 2. But for such matrix 
A+{M) = HiT). We are done. □ 

Note that since 7^(r) = A^{M), we actually obtain a basis of 'H{T), given 
by Tyj[x) foi' even x, and also a canonical filtration on 7^(r) with free finitely 
generated quotients. 
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